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Abstract 

We present the most recent steps undertaken to reform the introductory algebra-based course at The George 
Washington University. The reform sought to help students improve their problem-solving performance. Our 
pedagogy relies on didactic constructs such as the GW-ACCESS problem-solving protocol , instructional sequences 
and problem classification schemes that we have developed and implemented in our introductory physics course. 
These tools were designed to help advance students in two specific ways: 1) to improve their problem-solving 
performance and 2) to improve their attitudes towards learning physics. We organized traditional and research-based 
physics problems such that students experienced a gradual increase in complexity related to problem context, 
problem features and cognitive processes needed to solve the problem. The instmctional environment that we created 
is easily adaptable to any kind of curriculum and can be readily adjusted throughout the semester. To assess the 
students’ problem-solving performance, we created rubrics that assess key steps of physics problem solving. The 
Colorado Learning Attitudes about Science Survey (CLASS) was administered pre- and post-instmction to determine 
students’ shift in dispositions towards learning physics. The results show improvements in students’ problem-solving 
performance and in their attitudes towards learning physics. 
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Introduction 

Numerous research studies in the last 30 years have highlighted the inadequacy of traditional 
physics courses. Such courses are typically content-oriented and put little emphasis on cognitive 
processing. It has been shown that students who leave such courses tend to have incoherent 
physics knowledge and mediocre problem-solving abilities. Moreover, they have difficulties 
when engaged in high-level thinking, and they show little improvement in their attitudes towards 
science. Several reports (Czujko, 1997; NRC, 1996; AAAS, 1993) called explicitly or implicitly 
for reforming the instruction by exposing students to more process-oriented tasks that develop 
critical thinking and problem-solving abilities that are typically required at their future 
workplaces. 

ISSN: 1309-7202 


15 




EJPE 


European J of Physics Education Volume 5 Issue 1 2014 


Teodorescu et al. 


In recent years, a number of education research groups (Redish & Hammer, 2009; 
Beichner, Saul, Abbott, Morse, Deardorff, Allain, Bonham, Dancy, & Risley, 2007; Etkina & 
Van Heuvelen, 2007; McKagan, Perkins, & Wieman, 2006; Novak, Patterson, Gavrin, & 
Wolfgang, 1999; McDermott & The Physics Education Group of the University of Washington, 
1996; Crouch, Watkins, Fagen, & Mazur, 2007) have made sustained efforts to redesign the 
introductory physics courses in their departments. Their modifications focus on promoting 
conceptual understanding and higher-level thinking with the instruction. 

Prior to 2006, the GW algebra-based physics course (called Phys 11) had evolved from a 
traditional course to one that implemented technologically advanced methods, such as an 
electronic student response system coupled with Peer Instruction philosophy, the adoption of an 
individualized web-based homework system (LON-CAPA, www.loncapa.org) . computers in labs 
with data-acquisition software (PASCO Data Studio) and a special Physics Help Room equipped 
with terminals and staffed with physics majors, teaching assistants and course instructors to assist 
students in collaborative problem solving. 

With the existing infrastructure for the course already in place, we initiated an effort in 
2006 to refocus the curriculum towards problem solving. This reform was motivated by the 
unsatisfactory problem-solving performance of the students as reported by faculty, by the 
difficulties in embracing deep-thinking activities as frequently expressed in the course 
evaluations, and by marginal achievements with existing physics education research instructional 
strategies. Our efforts were informed by: 

• research on physics problem-solving expert-novice behavior (Maloney, 1994; Gerace & 
Beatty, 2005; Gerace, 2001) 

• cognitive science studies (Redish, 2003; Redish, 1994) 

• findings about students’ epistemological beliefs and their attitudes towards physics and 
learning physics (Adams, Perkins, Podolefsky, Dubson, Finkelstein, & Wieman, 2006; 
Elby, 2001) 

• educational psychology tools (Marzano & Kendall, 2007; Van Heuvelen, 1991) 

• skills and abilities required by employers of future graduates (NACE, 2008). 

Among existing research-based reformed courses, the GW curriculum is distinguished by 
the following characteristics: 

• It uses the New Taxonomy of Educational Objectives (NTEO) (Marzano & Kendall, 
2007) as a framework for teaching methodologies. 

• It is focused more on the design of the course structure rather than on creating new 
materials. 

• It seeks to combine existing research-based materials in a manner that fosters the 
development of critical thinking in different contexts. 

• It utilizes a multi-dimensional assessment. 

• It is easily adaptable to any kind of teaching environment. 
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The GW Course Reform Procedure 

Our initial attempts to reform the course sought to answer the following questions: 

1) What are the cognitive processes suitable for practice in this type of course at our 
university? 

2) Which processes need to be explicitly taught and which ones need only to be practiced? 

3) Where and how do the cognitive abilities need to be implemented within the curriculum to 
produce measurable effects in students ’performance? 

4) What training should be provided for Teaching Assistants? 

To answer these questions, during the Spring 2007 semester we introduced about 60 
research-based problems (problems that have been developed by physics education researchers) 
in the recitations and about 200 such problems in the homework. These problems were 
previously selected to trigger cognitive processes from all levels of NTEO. Professors’ 
observations from lectures were combined with Teaching Assistants’ observations from 
recitations, laboratories and Help Room sessions on a weekly basis. All consistent remarks were 
retained. In the end, the process of tailoring the course was based on: 

• students’ performance in warmups (Novak, Patterson, Gavrin, & Wolfgang, 1999), 
conceptests (Mazur, 1997), quizzes, recitations, homework and exams, 

• students’ behavior (comments, successes, difficulties) while solving recitation problems 
and homework problems, 

• students’ postings on the online discussion board in LON-CAPA, 

• students’ detailed comments about the curriculum materials and teaching methods in the 
mid- and end-semester evaluations, 

• professors’ and Teaching Assistants’ collective observations regarding students’ behavior. 

Taken together, these observations led to three main conclusions regarding instruction: 

• Referring to the NTEO, a realistic instructional goal for our class is practicing cognitive 
processes up to generalization of information and mental procedures. 

• All processes need to be made explicit to students while they solve the specific problems. 

• A gradual implementation of the different types of thinking seemed appropriate for our 
environment. 

With these observations in mind, we followed the timeline given in Table 1 to reform our course. 


Table 1 . The steps followed in reforming the GW algebra-based physics course. 


Semester 


Activity 


Fall 2006 + Spring 2007 


Development of new homework sets and problem-solving recitations 


Summer 2007 


Development of a new laboratory manual and refining homework and recitations 


Fall 2007 


Preliminary assessment of homework sets, recitations and lab activities 


Spring 2008 + Fall 2008 


Adjustment of lectures + Assessment 
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An important goal was to spend most of the time on the specific design rather than on 
creating new teaching materials. We aimed to find an instructional approach in which we could 
make use of the already-existing traditional and research-based materials. For this reason, the 
term “development” from Table 1 does not refer to creation of new tasks. Rather, it reflects our 
efforts to design the framework, search for suitable activities among existing teaching materials, 
implement them, and make adjustments to build a coherent teaching environment. 

The GW Course Curriculum Units 

The overarching learning objectives of our course relate to the physics content the students learn, 
the competencies they acquire, and their dispositions about science in general and physics in 
particular: we aim to help students improve their problem-solving performance and enhance their 
attitudes towards learning physics. The activities that students perform are organized into five 
curricular units, as shown in Figure 1: 

• Warmups - sets of reading quizzes that students need to answer before coming to class. 
These questions target very basic physics concepts that students should be able to 
understand after they read the textbook prior to the lecture. Their purpose is to provide the 
students with the necessary knowledge for the particular lecture. The warmups are made 
available to students through LON-CAPA fwww.loncapa.org) 24 hours before the lecture. 
This online system provides instant feedback, permits multiple attempts, and offers an 
electronic bulletin board for discussions. 

• Lectures - two 75-minute sessions per week that consist of three distinct components: 
1) assessment of students’ individual preconceptions with Peer Instruction techniques 
(Mazur, 1997), 2) clarification of the wrong preconceptions, and 3) problem solving. The 
classroom is equipped with an electronic student response system ( Turning Point). The 
final lectures focus on explaining to students why we focused on developing thinking 
skills, and where and how they can use the abilities they acquired in the future. 

• Recitations - one 90-minute meeting per week during which the students practice 
problem solving. The recitation sessions begin with a closed-book quiz based on 
problems taken from the homework and previous recitation problems. The quiz usually 
requires a symbolic solution. Its purpose is to provide students with feedback on their 
ability to solve problems, enabling them to gauge their performance in the course. The 
quiz also helps students develop the ability to solve symbolic problems. Later in the 
recitation, students are given cognitively complex physics problems that they solve 
following the GW-ACCESS problem-solving protocol and classification schemes that 
organize the various classes of problems typical for each chapter. The GW-ACCESS 
protocol and the classification schemes are detailed later in this paper. 

• Homework - two weekly sets, each containing 8-10 problems. The homework problems 
as well as the warmups are offered to students through LON-CAPA. 

• Laboratories - one 60-minute session per week in which students work in groups to 
devise and perform experiments. During each lab, students solve a real-world problem 
(taken or adapted from Heller & Hollabaugh, 1992). We follow the ISLE (Investigative 
Science Learning Environment) labs model (Etkina & Van Heuvelen, 2007). Table 2 
shows the abilities development targeted in parallel with the specific physics topic on a 
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weekly basis. During some of these activities, students use Data Studio, a software 
package from PASCO, to acquire and analyze data. 

The required textbook for the class was Giambattista, Richardson, & Richardson, 2005. 
The problems used, besides the required textbook, were selected or adapted from a variety of 
textbooks and research-based collections (Mazur, 1997; Cutnell & Johnson, 2006; Serway, 
Faughn, Bennett, & Vuille, 2005; Giancoli, 2005; Urone, 1998; Walker, 2004; O’Kuma, 
Maloney, & Hieggelke, 2000; Van Heuvelen & Etkina, 2006; McDermott, Shaffer, & University 
of Washington Physics Education Group, 2002). 



Figure 1 . The problem-solving learning cycle employed in the GW algebra-based physics course. 
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Table 2. The laboratory objectives related to content, abilities and cognitive processes. 


Week 

Physics Topics 

Abilities Targeted 

Type of experiment 
according to ISLE labs 
classification (Etkina 
& Van Heuvelen, 2007) 

Cognitive processes 
involved according 
to the NTEO 
(Marzano & 

Kendall , 2007) 

1 

Lab 1: Tutorial on 
precision and significant 
figures. Estimating and 
measuring time and 
distance. 

Making and recording 
measurements correctly. 
Evaluating the precision and 
accuracy of an instrument 

No specific lab type 

Retrieval 

2 

Lab 2: Vectors and 
Forces 

Testing two given hypotheses 
and eliminating the wrong one. 

Testing experiment 

Retrieval 

Comprehension 

Analysis 

3 

Lab 3a: Newton's 

Second Law and 
Kinematics 

Learning data acquisition with 
Data Studio software. Analyze 
kinematics graphs. 

Testing experiment 

Retrieval 

Comprehension 

Analysis 

4 

Lab 3b: Newton's 

Second Law and 
Kinematics continued 

Learning the meaning of 
"verifying a relationship". 

Testing experiment 

Retrieval 

Comprehension 

Analysis 

5 

Lab 4: Circular motion 

Learning how to derive 
mathematical relationships - 
when you have to derive the 
quantity you want from what 
you can actually measure. 

Testing experiment 

Retrieval 

Comprehension 

Analysis 

Knowledge 

utilization 

6 

Lab 5: Linear 

momentum 

Predicting and testing 
hypothesizes. 

Observational 

experiment 

Retrieval 

Comprehension 

Analysis 

Knowledge 

utilization 

7 

Lab 6a: Fluids 

Designing two experimental 
methods to measure the same 
quantity. 

Application experiment 

Retrieval 

Comprehension 

Analysis 

Knowledge 

utilization 

8 

Lab 6b: Fluids 
continued 

Comparing two experimental 
methods. Learn to evaluate them 
based on errors minimization. 

Application experiment 

Retrieval 

Comprehension 

Analysis 

Knowledge 

utilization 

9 

Lab 7a: Oscillations 
and waves 

Performing a complete 
investigation. 

Application experiment 

Retrieval 

Comprehension 

Analysis 

Knowledge 

utilization 

10 

Lab 7b: Oscillations 
and waves continued 

Writing a lab report. 

Application experiment 

Retrieval 

Comprehension 

Analysis 

Knowledge 

utilization 
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The GW Course Methodology 

In our view, introductory physics is seen as a “thinking-skills” program, and solving physics 
problems is a vehicle to promote cognitive growth and development that is intended to transfer to 
other areas. In a careful review of successful and unsuccessful thinking-skills programs, Mayer 
(1997) has proposed four issues as guidelines for thinking-skills courses: 

1) what to teach ( e.g ., problem solving as a single competency or a collection of component 
skills), 

2) how to teach (e.g. , focusing on the product or process), 

3) where to teach (e.g., in a general, domain-independent course or a domain-specific course 
such as physics), 

4) when to teach (e.g., after a set of basic skills has been mastered or before). 

Mayer and Wittrock (2006) found that thinking-skills courses are most effective under the 
following conditions: 

1) The curriculum focuses on one or more component skills that are developed separately, 
rather than treating problem solving and critical thinking as a single ability that the course 
tries to improve in general. 

2) The instructional methods focus on the problem-solving process itself, rather than on 
obtaining the right answer. 

3) The students learn and apply the skills within a specific domain, rather than across 
domains. 

4) Higher-level skills are developed and practiced even before students have automated the 
underlying basic skills. 

Our methodology is based on NTEO and on the Taxonomy of Introductory Physics 
Problems (TIPP) (Teodorescu, Bennhold, Feldman, & Medsker, 2013) that we developed for the 
purpose of this project. The taxonomy that we created establishes a relationship between physics 
problems and the cognitive processes that they trigger during the solving process. To help 
students perform better on problem solving, we developed three instructional strategies: GW- 
ACCESS protocol, instructional sequences, and classification. We used NTEO to establish the 
cognitive processes we want to exercise in the course and we used TIPP to select the specific 
physics problems that trigger these processes. The remaining part of this section will detail the 
theoretical basis, development and implementation of our teaching strategies. 

The GW-ACCESS Problem-Solving Protocol 

Most physics problem-solving protocols have their roots in mathematics problem solving or from 
the field of general problem solving (chess, games, puzzles). One of the pioneers in this area, 
Polya (1945), developed a four-step general approach for problem solving: understand the 
problem, devise a plan, carry out the plan, and look back, which was later adopted by Beichner 
and his colleagues (2007) and modified to become their GOAL protocol (Gather information, 
Organize and plan, perform the Analysis and Learn from your efforts) used in SCALE-UP 
(Beichner, Saul, Abbott, Morse, Deardorff, Allain, Bonham, Dancy, & Risley, 2007). Similar 
frameworks were used in later research that focused on students becoming better problem 
solvers. Reif and his colleagues (1976) used the following approach: description, planning, 
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implementation and checking for research purposes; Schoenfeld (1979) created a similar one for 
teaching purposes. Wright and Williams (1986) designed the WISE procedure: What’s happening 
(Identify givens and unknown, draw a diagram, identify the relevant physics principle), Isolate 
the unknown (select an equation, solve algebraically, look for additional equations if one is 
insufficient), Substitute (plug in both numbers and units), and Evaluate (check the reasonableness 
of the answer). Heller and Heller (1995) developed a five-step framework: Focus the problem, 
Describe the physics. Plan the solution, Execute the plan and Evaluate the answer. 

The protocols discussed above that are based on heuristics have long been recognized as 
standard approaches to systematic problem solving. The GW-ACCESS protocol builds on them, 
but incorporates our classroom experience into such schemes. While experimenting with these 
schemes throughout the years, we found that our students did not accept them naturally, often 
thought that they did not enhance their problem-solving ability, and frequently perceived them as 
obstacles rather than problem-solving aids. Moreover, they used them in an ineffective way. For 
instance, for steps like “Description” or “What’s happening” our students would just repeat parts 
of the problem statement showing no evidence that they can identify the key aspects particular to 
that problem. Such issues made us conclude that special attention should be paid to how we teach 
a protocol. The GW-ACCESS protocol has many features similar to other protocols, but it is 
accompanied by its own teaching methodology and assessment. To develop it, we first selected 
an appropriate taxonomy (NTEO) that organizes the component processes of problem solving. 
Then, we used expert-novice studies (Maloney, 1994; Gerace & Beatty, 2005; Gerace, 2001) to 
document the requirements and the implementation of each step. Table 3 shows the GW- 
ACCESS steps and the corresponding cognitive processes from NTEO. 

Table 3. The link between GW-ACCESS steps and NTEO. 


ACCESS stage 

Matching cognitive process in NTEO 

Assess the problem 

Categorization 

Create a drawing 

Symbolizing 

Conceptualize the strategy 

Integrating 

Execute the solution 

Executing 

Scrutinize your results 

Analyzing errors 

Sum up your learning 

Meta-cognition 


Following this further, we explain in detail the relationship between the GW-ACCESS steps and 
how they link with the NTEO and with expert-novice research (refer to Appendix A). 

A - Assess the problem 

Cognitive science research has shown (Chi, Feltovici & Glaser, 1981) that experts begin the 
problem-solving process by identifying the general category to which the problem belongs. 
Physicists first identify the general area of physics pertaining to the problem (mechanics, optics, 
etc.), then further specify one or more subcategories (energy conservation, momentum 
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conservation, etc.). Experts can perform such categorizations because their physics-specific 
knowledge is strongly interconnected and structured. For novices, such structures must be 
explicitly developed over time. 

C - Create a drawing 

Experts are able to create multiple representations of the information and translate between those 
representations in the problem-solving process, with the standard representations in physics being 
diagrams, words, equations, plots, tables of numbers, etc. Faced with a word problem, experts 
will usually draw a diagram that represents the available information and then proceed to develop 
a strategy that identifies the equations needed to solve the problem. In the NTEO, the process of 
developing and translating between knowledge representations is called symbolizing. 

C - Conceptualize the strategy 

Unless a solution is obvious to them, experts tend to formulate a strategy to solve a problem. Part 
of formulating the strategy is to separate the critical from the non-critical components of the 
information provided, and to select the appropriate algorithms needed to proceed towards a 
solution. In the NTEO, this process of selecting and identifying the relevant information and 
mental procedures is called integrating. 

E - Execute the solution 

This is the step that novice students usually begin with when solving physics problems: “What is 
the right equation I need to use in order to plug in the numbers and get my result?” According to 
the NTEO, executing is a lower-level thinking process that only involves performing algorithms 
or procedures and does not necessarily involve the understanding of what it means to perform 
these procedures. Novices will tend to get stuck at this stage when their selected algorithm does 
not lead to the solution of the problem. 

S - Scrutinize your results 

Experts will perform a qualitative analysis of their result to verify that the answer is reasonable. 
This involves checking reasonable assumptions related to the physics information, making 
estimates and looking for errors. This is a higher-level thinking ability referred to as analyzing 
errors in the NTEO. 

S - Sum up your learning 

Finally, experts reflect on their own problem-solving process and ability, and identify their 
strengths and weaknesses. This form of meta-cognitive processing is present not only at the end, 
but to some degree during the entire problem-solving process; it involves self-monitoring of 
efficacy, clarity and accuracy by the expert. Novices usually suffer cognitive overload and have 
already expended all of their mental resources on the problem-solving process. Thus, meta- 
cognitive reflection during the problem-solving process usually develops slowly and needs to be 
explicitly encouraged and nurtured. 

To address each step of the ACCESS protocol, students have been provided with specific 
questions accompanied by succinct instructions. Appendix A shows a version of the protocol that 
we tested in the classroom. This scheme is iterated and enhanced as we probe students’ 
understanding of the language used to describe each step. 

To gradually implement the GW-ACCESS plan in the curriculum, we used a complex 
syllabus (shown in Figure 2) that outlines both the physics content and the highest cognitive 
process that is to be developed and practiced in a particular week. Here, we refer to the hierarchy 
of cognitive processes presented in the NTEO - the reader should understand that during each 
week we primarily exercised the cognitive process listed in Figure 2 and all the processes below 
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it in the NTEO. Note that students will not work with the complete problem-solving protocol 
until halfway through the semester. The problems and exercises used in the course are matched 
with the cognitive processes that are targeted for development by using TIPP - the Taxonomy of 
Introductory Physics Problems (Teodorescu, Bennhold, Feldman & Medsker, 2013). 


Week 

Physics content 

The highest cognitive process practiced 



Information 

Mental Procedures 

1 . 

Introduction - What is physics? 

Introduction - What is a thinking process? 

1 . 

Forces and vectors 

Integrating 

Executing 

2. 

Newton's Laws 

Symbolizing 

Integrating 

3. 

ID Kinematics 

Matching 

Integrating 

4. 

2D Kinematics 

Classifying 

Matching 

5. 

Review and Midterm 1 

Review and 

1 Midterm 1 

6. 

Energy conservation 

Analyzing errors 

Classifying 

7. 

Collisions and momentum 

Analyzing errors 

Analyzing errors 

8. 

Rotational motion 

Using ACCESS 

9. 

Buoyancy and fluid flow 

Using ACCESS 

10. 

Review and Midterm II 

Review and Midterm II 

11. 

Oscillations 

Using ACCESS 

12. 

Waves and sound 

Using ACCESS 

13. 

Trying it all together 

Using ACCESS 

14. 

Putting all together 

Why did you learn this content and these skills 
Where can you use the physics you learned and 
the abilities you acquired 

15. 

Cumulative final exam 

Cumulative final exam 


Figure 2. The first semester of the algebra-based introductory physics sequence. The weekly outline 
specifies both the physics content and the thinking process to be practiced. Here by “information ” 
we mean declarative knowledge and by “mental procedures ” we mean procedural knowledge. 


The Teaching Method Used to Help Students Achieve Coherent Knowledge Structures 

Helping students acquire locally and globally coherent physics knowledge is a focal point 
of our problem-solving training. This translates into helping them see relevant details of the 
knowledge within a unit, as well as the “big picture” aspect of the material. By locally coherent 
knowledge we mean that students perceive different pieces of knowledge as being closely related 
to each other. By globally coherent knowledge we mean “that students recognize sets of locally 
coherent knowledge as appropriate for a certain problem and useful together” (Sabella & Redish, 
2007). The lack of knowledge coherence in novices has been pointed out by physics education 
research (Redish, 2003) and expert-novice research (Maloney, 1994; Gerace & Beatty, 2005). We 
have chosen to assist the students in this matter by using the cognitive processes of 
categorization and classification. These processes have been identified by physics education 
research (Beatty, Gerace, & Dufresne, 2006) as “advanced habits of mind that experts possess 
and students should develop”. In addition, problem classification schemes have been proven to be 
effective in physics instruction (Van Heuvelen, 1991). 
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For each chapter we created classifications of physics problems according to the physics 
laws or concepts that are required to solve them. As an example, Figure 3 shows the scheme we 
use for the unit on Fluids. 



Figure 3. The problem classification scheme used for the unit on Fluids. 


These classification schemes are given to students during the recitation to help them answer 
the “Assess the problem” step in the GW-ACCESS protocol. Students are encouraged to use the 
schemes for the homework problems and are expected to be able to easily identify the physics 
principles involved in the problem after repeated use of those schemes. The classification 
schemes have multiple pedagogical functions: a) to implicitly direct students’ focus towards the 
deep features of the physics problems and a procedure-oriented thinking; b) to help initiate the 
problem-solving process by making the activation of knowledge easier; c) to help students build 
locally coherent entities of knowledge; d) to raise students’ level of motivation by explicitly 
showing them that the diversity of physics problems is, to some extent, limited and relatively 
well-defined. In the middle and at the end of the semester, we hold review sessions using more 
complex charts that link the chapters’ content. Figure 4 shows, for example, the chart that 
summarizes Chapters 2-5 (featuring Newton’s 2 nd Law and kinematics). 
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Figure 4. The problem classification scheme that is used to summarize the types of physics 
problems for Chapters 2-5 (which includes kinematics and forces). 


The Teaching Strategy for Helping Students Solve Problems Featuring Different Contexts 

Students’ difficulties with physics problem solving have been extensively studied in recent years. 
Several results seem to be consistent across different studies: a) students perceive the same 
problem offered to them in symbolic and numeric forms differently, and consequently perform on 
it differently (Torigoe & Gladding, 2007), b) students perceive conceptual and numerical 
problems that involve the same physics differently, and consequently perform on them differently 
(Sabella, 1999), c) surface features (i.e. the system presented in the problem) and deep features 
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(i.e. the physics principle involved in the problem) are important aspects that not only 
differentiate the experts from novices (Chi, Feltovici, & Glaser, 1981) but also, in some 
circumstances, determine the extent to which students choose a certain problem-solving approach 
(Mestre, Dufresne, Gerace, & Hardiman, 1993). Furthermore, a natural implication for teaching 
is to diversify contexts and surface and deep features in the instruction. We used a small number 
of logical constructions, which we call instructional sequences, to create a variety of different 
contexts, problem features and corresponding thinking processes. 

The instructional sequences are entities that combine traditional and research-based physics 
problems such that, during each week as students learn a new topic, they also gradually 
experience a new type of thinking that involves certain contexts, problem features, and cognitive 
processes. After students solve the problems chosen according to a certain instructional sequence, 
the instructor leads a guided discussion to convey to students that it is not very effective to think 
primarily of surface features during problem solving because the solutions should instead be 
triggered by deep features. In addition, we use instructional sequences to link the different 
curricular units, thus addressing a common student complaint that the various course elements, 
such as textbook readings, lecture materials, homework problems and lab exercises, appear 
disjointed and unrelated to each other. Four types of instructional sequences have been created 
and are explained below. An overview of all the instructional sequences and the characteristics 
of physics problems they involve is shown in Table 4. The table explains what features and 
contexts were varied within each instructional sequence. 

Figure 5 shows an example of Instructional sequence 1, while Appendix B presents other 
types of instructional sequences that we use. Instructional sequence 1 blends deep features and 
contexts maintaining the same surface features. We used it in homework and in recitations. 
Figure 5 shows an example taken from recitation. The key insight here is that from week 2 to 
week 6 we gave students the same problem involving a skier and an inclined plane which was 
altered to deal with different physics principles. Also, in weeks 2 and 4 the problems were in 
symbolic form, while the one from week 6 was numeric. 


Table 4. Summarized description of the blending of surface features, deep features and contexts in 
the instructional sequence. Note: Instructional sequence 4 is not related to the other sequences. 
It is specifically designed to allow cognitive processes ’ monitoring. 


Instructional sequence 

Surface Features 

Deep Features 

Contexts 

Curriculum Units 

Instructional sequence 1 

same 

different 

different 

recitation, homework 

Instructional sequence 2 

same 

different 

same 

homework 

Instructional sequence 3 

different 

same 

different 

all 

Instructional sequence 4 

targets specific cognitive processes 

recitation 
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Figure 5. An example of Instructional sequence 1 taken from recitation. Note the same 
surface features (inclined plane) but different contexts and deep features. 


Figures 6a, 6b and 6c show the types of problems that we used in the reform curriculum. 
As seen in Figure 6a, while the majority of our problems are traditional ( e.g. typical textbook 
problems), a substantial number of these problems are research-based. The research-based 
problems target various cognitive processes that are illustrated in Figure 6b. The definitions of 
these processes are the ones given in Teodorescu, Bennhold, Feldman, & Medsker, (2013). The 
processes shown are the highest ones involved in the research-based problems. We sought to 
offer students a wide variety of problems that deal with real-world situations. Figure 6c shows 
how we distributed those problems across the curriculum. 
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Figure 6. The problems used in recitation and homework in the reformed algebra-based curriculum. 
Representation tasks ask students to represent the information in various ways. Integrating problems ask 
students to explicitly or implicitly identify what is relevant and what is irrelevant in a problem text. 
Ranking tasks ask students to compare, contrast, match and rank physical quantities, scenarios or 
solutions. Analyzing error problems ask students to either identify errors in the problem's text or solutions 
or discuss the reasonableness of the answers. Generalization problems ask students to create generalizing 
statements based on a sequence of tasks (Teodorescu, Bennhold, Feldman, & Medsker, 2013). 


The Assessment of the GW Reformed Curriculum 

To evaluate the efficacy of our pedagogy, a multi-dimensional assessment methodology was 
used, as outlined in Table 5: 

• A problem-solving assessment based on rubrics was created specifically for this project, 

• Force Concept Inventory was administered pre- and post-instruction, 

• Colorado Learning Attitudes about Science Survey (CLASS) was also given to students 
pre- and post-instruction. 

Table 5. The outcomes assessment methods 


Course outcomes 

Teaching methods 

Assessment 

Help students acquire 
conceptual understanding 

Peer Instruction 

Force Concept Inventory 

Help students improve their 
problem-solving performance 

GW-ACCESS protocol 

Rubrics 


Problem classification schemes 
that summarize the information 
and mental procedures 



Instructional sequences 


Help students improve their 
attitudes towards learning 
physics 

Allotted lecture time to point out 
when, where and how students 
can use both the content and the 
abilities learned 

Colorado Learning Attitudes 
about Science Survey 
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The Assessment of Students’ Problem-Solving Performance 

Rubrics are a common tool to provide formative assessment of student performance. As scoring 
devices that include descriptions of different levels of accomplishment, rubrics are associated 
with numeric scores, with the highest level reflecting perfect completion of a task or total 
proficiency in the area being evaluated (Brookhart, 1999). They provide a relatively unbiased, 
quantitative way to examine different steps in student work. For example, rubrics allow us to 
score a student’s free-body diagram regardless of whether the student obtained the correct answer 
to the entire problem. For the purpose of this project, we have chosen rubrics as a 
straightforward method of measuring the improvement of student performance in component 
processes as applied to solving a physics problem, independent of their performance in other 
parts of the problem-solving process. 

In-depth work with assessment rubrics has been done by the Physics and Astronomy 
Education Research (PAER) group at Rutgers University. The group developed rubrics for the 
assessment of seven scientific abilities, which they define as the most important procedures, 
processes, and skills scientists use when constructing scientific knowledge and when solving 
experimental problems (Etkina, Van Heuvelen, White-Brahmia, Brookes, Gentile, Murthy, 
Rosengrant, & Warren, 2006). Each rubric has a range of scores from 0 to 3, with 0 (missing), 1 
(inadequate), 2 (needs improvement), or 3 (adequate). We followed their idea and designed 
rubrics to evaluate the different steps involved in solving a physics problem and we applied them 
to the problem below. We used rubrics to grade students’ quizzes throughout the semester; in 
addition, we gave them the problem below after Newton’s 2 nd Law was learned early in the 
semester and also in the final exam at the end. Students were asked to solve it following the GW- 
ACCESS protocol. Separate rubrics were used to grade the four parts: Assess the problem, Create 
a drawing, Conceptualize the strategy and Execute the solution parts. The GW-ACCESS parts 
were graded with rubrics developed following the model of Etkina and her colleagues (2006). 


Assessment Problem : Two blocks of masses Mi = 8 kg and M 2 = 20 kg are connected as 
shown in Figure 7 below. The two inclined planes are frictionless. The string is 
inextensible and both the pulley and the string have negligible mass. Knowing the angles 
6= 60° and f = 30°, find the acceleration of each block and the tension in the string. 



Figure 7. Modified Atwood machine. 
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Figures 8-11 illustrate the rubrics used to evaluate students’ abilities and the positive shifts 
in students’ performance for each of the abilities targeted. The gain indicated in the figures is the 
difference between the post- and pre- distribution means. The grading was performed by one 
grader and therefore no inter-rater reliability testing was necessary. 

The results for Assess the problem step are shown in Figure 8. They indicate the highest 
gain in students’ performance as compared with the other abilities. This may suggest that, among 
the abilities taught, this one seems to be the best learned by the students. This can also be an 
indication of the effectiveness of our classification schemes. Also, the pre-test mode distribution 
is 2, as compared to the pre-test mode distribution for the other abilities, which is 1. This 
indicates that, even before instruction, students seem better prepared to assess a problem than to 
create the corresponding drawing and strategy and to solve it. The results for Create a drawing 
step are shown in Figure 9. They indicate the lowest gain in students’ performance as compared 
with the other abilities. This may suggest that, among the abilities taught, this one seems to be the 
hardest to learn for students. The results for Conceptualize the strategy step are shown in Figure 
10. They indicate the second highest gain in students’ performance. This gain, associated with the 
low pre-test distribution mode (which is 1), may imply that students made good progress in 
learning this ability. The results for Execute the solution step are shown in Figure 11. They 
indicate a moderate gain in students’ performance. It should be noted that the assessment problem 
we used is considered a difficult problem for algebra-based physics students. The moderate gain 
we achieved can be due to the difficulty of this problem. 


Scientific Ability 

Missing (0) 

Inadequate (1) 

Needs improvement (2) 

Adequate (3) 

Is able to assess 
(categorize) the 
problem according 
to deep features (the 
underlying physics 
concepts or 
principles). 

No attempt 
is made to 
assess the 
problem. 

The problem is 
assessed 
incorrectly or 
according to 
surface features. 

The problem is assessed 
correctly but too 
generally (according to a 
concept and not a 
principle). 

The problem is 
assessed according 
to correct physics 
concepts and/or 
principles. 



Figure 8. The rubric used to evaluate the Assess the problem ability and students ’ scores. 
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Scientific Ability 

Missing (0) 

Inadequate (1) 

Needs improvement (2) 

Adequate (3) 

Is able to draw the 
corresponding 
free-body diagram 
correctly. 

No FBD is 
constructed. 

FBD is constructed 
but contains major 
errors such as 
incorrect force 
vectors, wrong 
direction, extra 
incorrect vectors, 
or missing vectors. 

FBD contains no errors in 
vectors but lacks a key 
feature such as labels of 
forces or axes are missing. 

The FBD contains no 
errors and each force 
is labeled so that it is 
clearly understood 
what each force 
represents. Axes are 
present. 


"Create a drawing" 
Fall 2008 


42 T 



0 12 3 

Grade 


Figure 9. The rubric used to evaluate the Create a drawing ability and students ’ scores. 


Scientific Ability 

Missing (0) 

Inadequate (1) 

Needs improvement (2) 

Adequate (3) 

Is able to identify 
the necessary steps 
required to solve 
the problem. 

No attempt 
is made to 
write a 
strategy. 

The strategy is 
either too 
generic or 
wrong. 

The strategy contains 
some correct steps 
pertaining to the problem 
but critical information is 
missing. 

The strategy contains 
the necessary steps 
and critical 
information to solve 
the problem. 


"Conceptualize the strategy" 

Fall 2008 



Grade 


Figure 10. The rubric used to evaluate the Conceptualize the strategy ability and students ’ scores. 
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Scientific Ability 

Missing (0) 

Inadequate (1) 

Needs improvement (2) 

Adequate (3) 

Is able to execute 
the necessary 
steps required to 
solve the problem. 

No attempt is 
made to 
execute the 
solution 
symbolically. 

The solution contains 
mostly wrong equations 
or only numerical 
equations. No attempt is 
made to solve the 
problems symbolically. 

The solution contains 
some good equations, but 
some equations are 
wrong. 

The solution 
contains all 

correct 

equations. 


"Execute the solution" 
Fall 2008 


■ Pre 
□ Post 


0 12 3 

Grades 



Figure 11. The rubric used to evaluate the Execute the solution ability and students ’ scores. 


Table 6 shows the positive shifts obtained after the instruction for all these abilities. They 
are statistically significant (p =.0001, Wilcoxon nonparametric test) and suggest that students’ 
problem-solving behavior has improved. Student post-instruction performance was the highest on 
the Assess the problem ability (Figure 8) and the lowest on the Execute the solution ability 
(Figure 11). The highest gain was achieved for Assess the problem ability and the lowest for 
Create a drawing ability (Figure 9). Moderate improvements were found in the other abilities. 


Table 6. The class pre and post means and the absolute gain (post-pre) for each ability. 


Ability 

Pre 

Post 

Gain 

Assess the problem 

1.93 

2.72 

0.79 

Create a drawing 

1.46 

1.90 

0.44 

Conceptualize the strategy 

1.00 

1.71 

0.71 

Execute the solution 

0.77 

1.45 

0.68 
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The Assessment of Students’ Attitudes About Learning Physics 

It is well known that what students believe about physics as a science and what they expect from 
their physics courses determine their attitude and motivation towards the process of learning 
physics. Ultimately, those factors influence their overall achievement in a physics course. The 
most co mm on instruments for measuring students’ expectations, beliefs and attitudes in physics 
courses are the MPEX (Redish, Steinberg, & Saul, 1998), VASS (Halloun & Hestenes, 1996), 
CLASS (Adams, Perkins, Podolefsky, Dubson, Finkelstein, & Wieman, 2006) and EBAPS (Elby, 
2001). As stated at the beginning of this paper, one of the learning objectives for our introductory 
physics course is not only to enhance students’ problem-solving abilities but also to influence 
their attitudes about learning physics. One of the recently developed instruments to measure such 
beliefs and attitudes is the Colorado Learning Attitudes about Science Survey (CLASS). We 
administered this survey in Fall 2008 and the results are shown in Table 7. Overall, they indicate 
a generally positive trend, compared to the negative trend reported in the literature. These shifts 
suggest that students exhibit more expert-like attitudes at the end of the instruction compared to 
the beginning, in six out of eight categories. Traditional lectures have been reported to generate 
shifts on the overall CLASS of-8.2% to +1.5% for calculus-based courses, shifts of 1% for non¬ 
science majors and -9.8% for algebra-based courses for premedical students (Perkins, Adams, 
Pollock, Finkelstein, & Wieman, 2005). 


Table 7. Results for CLASS survey administered in Fall 2008. Positive gains have been 
achieved, indicating changes in students’ views towards more expert-like views. (Standard 
error is given in parentheses.) 


Category 

Pre 

Post 

Shift (Diff. of averages) 

Overall 

63.9 (1.88) 

69.5 (1.93) 

5.6 (1.3) 

Real-world connections 

68.8 (3.84) 

83.1 (3.20) 

14.2 (4.0) 

Personal interest 

65.6 (3.33) 

71.8(3.43) 

6.2 (3.0) 

Sense-making effort 

74.9 (2.43) 

73.0(2.70) 

-1.9 (2.3) 

Conceptual understanding 

59.2 (3.28) 

62.6 (3.80) 

3.4 (3.1) 

Applied conceptual understanding 

46.3 (3.00) 

56.5 (3.47) 

10.2 (3.1) 

Problem solving (general) 

74.2 (2.82) 

75.8 (2.85) 

1.5 (2.4) 

Problem solving (confidence) 

71.5 (3.33) 

76.5 (3.71) 

5 (3.5) 

Problem-solving (sophistication) 

52.5(3.81) 

59.5 (3.64) 

7(3.4) 


In the course evaluations, students ranked the usefulness of the lectures as 4.28/5 and of the 
recitations as 3.34/5, which is an improvement as compared to previous semesters when the 
usefulness of the lectures was ranked as 3.40/5 and of the recitation as 2.76/5. 
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Conclusions 

We presented in this paper the reform approaches that we created to enhance students’ problem¬ 
solving performance and improve their learning attitudes in our introductory algebra-based 
curriculum. Several comments are worth mentioning here about the course. To promote 
systematic problem solving, we had to challenge the students with complex problems. The 
appropriate level of complexity was determined according to the specifics of our student 
population. Problems that were too difficult discouraged students, while easy ones sent the 
message that they had enough knowledge and did not need to learn anything new. Facing hard 
problems, students tend to be discouraged in the beginning; therefore the instructor and the 
teaching assistants have a critical role in helping them overcome this initial frustration in a 
constructive way. 

A second issue is related to problem selection and implementation of the GW-ACCESS 
protocol. To help students accept and learn how to use this protocol in a natural way, we not only 
broke it into steps and gradually offered it to students, but we also made sure that the problems 
we used in recitation in a particular week reflected the cognitive processes that we wanted the 
students to learn in the GW-ACCESS step. Furthermore, this was reinforced by the exam 
problems, which had formats similar to the recitation problems. 

A third aspect, related to the problems used, is that we tried to offer problems that have 
features similar to problems previously solved by students. This was done using the instructional 
sequences. An expert will easily find the common features of the problems belonging to each of 
the instructional sequences presented. However, from our experience, students have to be 
constantly prompted and explicitly shown the link between the problems. Otherwise, they 
perceive them as very different problems with no common features. 

Regarding student dispositions, as the semester went on, we explicitly addressed the need 
for learning problem solving. In this way, we aimed to help students appreciate their efforts, thus 
increasing their motivation and self-esteem. We did not initiate such discussions early in the 
semester, because we found that this has a detrimental effect on students’ attitudes. Our semester 
starts with the entire focus on physics in general and after four or five weeks we begin to spend 
10-15 min of lecture time to talk in more depth about problem solving. We discovered the 
beneficial impact of a final lecture that clearly explains the wide range of employment 
opportunities that require introductory physics knowledge, the demands for problem-solving 
abilities stated in political, educational, and social reports; and the problem-solving abilities that 
are becoming mandatory for employment according to national surveys. In addition, in our final 
lecture, we remind the students when and how we practiced all these abilities in our course. We 
conclude with everyday examples in which variations of the GW-ACCESS protocol can be used 
to solve complicated and demanding life problems. 

Teaching Assistant (TA) training is a key component of success in our classes. TAs do not 
automatically understand the philosophy behind this course, especially when they teach it for the 
first time. While the physics content is usually familiar to them, they have difficulties 
understanding why we use the specific problems and not the standard textbook problems. Open 
dialog, continuous assistance and clear explanations of the pedagogy, along with precise teaching 
guidance, are often enough to help them become comfortable and proficient with this instruction. 

Regarding the assessment of our course, it should be noted that our design made possible a 
detailed evaluation of the key steps involved in basic physics problem solving. We consider the 
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traditional grading method based on “correct answers” important. However, we often observed 
that many students perform various steps related to a problem correctly, even when they do not 
manage to obtain the right answer. We hope that our tool, together with other existing 
assessments (Adams & Wieman, 2006; Docktor, 2009; Cummings & Marx, 2010), will enable 
the instructors to better evaluate students’ problem-solving performance. 

The results we obtained in all the assessments are encouraging: they show improvement in 
student problem-solving abilities and their attitudes. CLASS administered pre- and post¬ 
instruction revealed that student’ beliefs have changed in a positive way. Students seem to realize 
the connection between the physics they learned and the real world better after the instruction and 
believe that they can apply the concepts they learned. During the instruction, they built 
confidence in their problem-solving ability and their interest was stimulated. 

Future work will seek to continue the refinement and assessment of this course. Using the 
existing assessments, we intend to build a longitudinal database in order to document the 
successes and limitations of our methodology. In addition to the ones we used, more assessments 
have to be implemented to evaluate aspects of our instruction that have not been documented at 
this stage, including the following: 

1. The remaining GW-ACCESS steps {Scrutinize your results and Sum up your learning ) for 
which the rubrics are not yet finalized; 

2. An analysis of the connections between students’ responses on each step of the GW- 
ACCESS protocol, to judge the extent to which they are grasping the whole problem¬ 
solving process; 

3. The laboratory activities (which can be evaluated using the method described by Etkina 
and her colleagues (2006); 

4. The coherence of students’ knowledge (which can be evaluated using the method 
described by Sabella (1999)); 

5. Students’ meta-cognitive behavior, which we recently started to monitor in one pilot lab 
section. 

The refinement of the course will aim to help students achieve more significant improvements 
in their conceptual understanding. 
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APPENDIX A: The GW-ACCESS Protocol 

A - Assess the problem 

What is your first impression of the problem? Can you verbalize what the problem is actually asking you to do? 

• Identify the physics principles needed to solve the problem. Physics principles are described by words like 
momentum conservation, energy conservation, Newton’s 2 nd Law , etc. If principles are not involved in the 
problem, identify the concepts that are needed to solve the problem. Physics concepts are described by 
words like torque, momentum , work , etc. 

C - Create a drawing 

Translate the words of the problem into a diagram or a picture that contains clues for how to solve the problem. A 
drawing can be: 

• A free-body diagram (FBD) that displays the forces that act on an object. 

• A diagram that displays the motion of an object chronologically. 

• A diagram that compares the initial and final states of an object, e.g ., for energy and momentum 
conservation. 

C - Conceptualize the strategy 

Develop a strategy that outlines the steps you need to follow to solve the problem. The signature of a good strategy 
is that you could give your strategy to someone else who could then solve the problem. Thus a good strategy 
addresses the following questions: 

• Does the problem need to be broken into parts, either because there are several objects or because the same 
object experiences different kinds of motion? 

• What physics principles apply to each part and how do they apply to the specifics of the problem? 

• What is the key insight that is needed to solve the problem or each part of the problem? 

• How do the parts link together to give you the solution of the whole problem? 

• What are the relevant equations that represent the physics principles that you’ve outlined? 

• What are the known and unknown quantities of the problem? 

E - Execute the solution 

Now that you’ve done the setup of the problem solution, you have to go through the steps you’ve outlined to reach 
the answer. 

• Solve the equations algebraically for the unknown variable that you are looking for. Show all your 
algebraic steps explicitly in symbolic form (no numbers yet!). Make sure that your unknown quantity is 
expressed in terms of known variables. 

• Plug in the numerical values given in the problem to obtain your final answer. This may involve getting an 
intermediate result and then using that number in a subsequent part. 

S - Scrutinize your results 

You need to do a “reality check” to make sure that your answer is sensible. 

• Check your units. 

• Are the numerical values you got reasonable? 

• Compare the situation presented in the problem with a real-world situation. 

• Try some limiting cases. 

S - Sum up your learning 

This is the time to reflect on the purpose of this exercise. This helps put the exercise into a larger context, in the 
overall framework of physics, for one thing, and also in the ongoing evolution of your own learning. 

• How did this problem link to a physics concept which is a part of a greater whole? 

• How did this problem help you to improve your problem-solving skills? How did this problem apply the 
laws of physics to a real-life situation? 
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APPENDIX B: Instructional Sequences 

Instructional sequence 2 featured in Figure 12 combines problems having the same surface 
features and contexts but different deep features. It was used in recitation and homework. The 
system presented is always the bungee jumper and the context is always numerical, but the 
physics principle involved in each of the problems is different. 



Figure 12. An example of Instructional sequence 2 taken from homework. Note the same 
surface features (bungee jumper) and context (numerical) but different deep features. 


Instructional sequence 3 shown in Figure 13 brings together problems having the same deep 
features but different surface features and contexts. We used it to link two or more curriculum 
units. Note how a symbolic problem related to Newton’s 2 nd Law given in week 2 in recitation is 
followed by numeric problems involving Newton’s 2 nd Law and the same free-body diagram 
given in the homework. All these problems need similar free-body diagrams. 
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Figure 13. An example of Instructional sequence 3 that combines problem from recitation 
and homework. Note the same deep features (Newton’s 2 nd Law, the same free- body 
diagram), but different surface features (helicopter and train) and contexts (symbolic and 
numeric, abstract and real-world). 


Instructional sequence 4 was designed to have students practice specific cognitive processes in 
addition to the thinking triggered by the above progressions. The cognitive processes targeted 
are: filtering the relevant information, creating a strategy, problem classification, ranking of 
different physical quantities, comparison and analysis of different phenomena, analyzing 
procedural errors ( i.e ., identify errors in wrong solutions offered) or informational errors ( i.e. 
numbers, premises). Figure 14 shows an example in which students exercise processes like 
creating a strategy, ranking of information and analyzing errors related to information. We used 
this instructional sequence in the recitation sessions. 
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Figure 14. An example of Instructional sequence 4 taken from recitation. Note how the cognitive 
processes targeted increase in complexity as more content is taught from week 2 to week 6. 
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APPENDIX C 

Proposed solution for the Example assessment problem. The solution that is considered to be 
correct is presented below. 

A - Assess the problem 

This is a Newton’s 2 nd Law problem. 

C - Create a drawing 


y 



C - Conceptualize the strategy 

I treat each object separately. I apply Newton’s 2 nd Law for each object: ZF X = ma ZF, = 0 


Key insights : Both objects move with the same acceleration. The same tension acts on both objects. The 
coordinate system should be tilted for convenience. 

Known : Mi=8 kg, M 2 =20 kg, 0= 60°, a= 30° 

Unknown : T, a 


E - Execute the solution 


(MO ZF X = M,a 


T-M^g smQ = M^a (A) 


(MO 2F X = M 2 a 


M 2 g sin a -T = M 2 a (A) 


T = M,gs\r\Q + M,a (B) 



a = 1 ,07ml s 2 (C) 


T = M x g sin 6 + 



T = 76.57 N (C) 
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